Rational invariants of ternary forms under the orthogonal group

In this article we determine a generating set of rational invariants of minimal cardinality for the action of the orthogonal group O3 on the space R[x, y, z]2d of ternary forms of even degree 2d. The construction relies on two key ingredients: On one hand, the Slice Lemma allows us to reduce the problem to dermining the invariants for the action on a subspace of the finite subgroup B3 of signed permutations. On the other hand, our construction relies in a fundamental way on specific bases of harmonic polynomials. These bases provide maps with prescribed B3-equivariance properties. Our explicit construction of these bases should be relevant well beyond the scope of this paper. The expression of the B3-invariants can then be given in a compact form as the composition of two equivariant maps. Instead of providing (cumbersome) explicit expressions for the O3-invariants, we provide efficient algorithms for their evaluation and rewriting. We also use the constructed B3-invariants to determine the O3-orbit locus and provide an algorithm for the inverse problem of finding an element in R[x, y, z]2d with prescribed values for its invariants. These are the computational issues relevant in brain imaging.